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CONTROL APPARATUS 

REFERENCE TO RELATED APPLICATIONS 

The present application is related to U.K. Patent Application Serial No. 
GB0407453A0 filed April 1, 2004 and entitled "CONTROL APPARATUS", from which 
Convention priority is hereby claimed under 35 U.S.C. 1 1 9a. This application also is related 
to copending U.S. Patent Application Serial No. 1 1/006,486, filed December 6, 2004. The 
disclosures of both of the aforesaid patent applications is hereby incorporated herein by 
reference. 

FIELD OF THE INVENTION 

The present invention relates to interfaces generally and more particularly to user 
interfaces and control apparatus and methodologies for and for use with computerized 
devices, such as mobile communicators, televisions and personal computers. 

BACKGROUND OF THE INVENTION 

The following published patent documents, the disclosures of which are hereby 
incorporated by reference, are believed to represent the current state of the art: 
Great Britain Patent Numbers: GB2299856 and GB2289756, 
European Patent Number: EP05721 82, 

PCT Patent Application Publication Numbers: WO02/043045 and 
WO95/02801, and 

US Patent Numbers: 6,094,188; 6,081,255; 5,926,168; 5,892,501; 5,448,261; 
5,227,985; 5,949,402; 5,959,617; 5,122,656; 5,506,605 and 4,320,292. 
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SUMMARY OF THE INVENTION 

The present invention seeks to provide improved control apparatus and 
methodologies for and for use with computerized devices, such as mobile communicators, 
5 televisions and personal computers. 

There is thus provided in accordance with a preferred embodiment of the present 
invention an interface apparatus including a panel defining at least one edge, at least one 
detector arranged along the at least one edge of the panel and an electromagnetic radiation 
beam emitter operative to direct at least one beam of electromagnetic radiation onto the 
1 0 panel from a variable distance and at a variable angle, the panel being operative to transmit 
electromagnetic radiation from the at least one beam impinging thereon to the at least one 
edge thereof, for detection by the at least one detector, the panel being operative to 
generally attenuate the electromagnetic radiation passing therethrough to the at least one 
edge as a function of the distance traveled by the electromagnetic radiation through the 
1 5 panel, whereby the at least one detector is operative to provide at least one output which can 
be used to determine the variable distance and the variable angle. 

In accordance with a preferred embodiment of the present invention the panel 
includes a display. Preferably, the panel includes a mobile telephone display panel. 
Alternatively, the panel includes a hand-held computing device display panel. As a further 
20 alternative, the panel includes a television display panel. As yet a further alternative, the 
panel includes an input pad panel. 

In accordance with another preferred embodiment of the present invention the at 
least one detector includes a generally linear array of detectors. Preferably, the at least one 
detector is capable of detecting the electromagnetic radiation at predetermined frequencies 
25 in at least one of visible and non- visible ranges. 

In accordance with still another preferred embodiment of the present invention the 
electromagnetic radiation beam emitter is operative to provide a generally conical beam. 
Alternatively or additionally, the electromagnetic radiation beam emitter is operative to 
provide a plurality of beams. Preferably, the electromagnetic radiation beam emitter is 
30 operative to provide at least one generally collimated beam. Alternatively, the 
electromagnetic radiation beam emitter is operative to provide at least one beam having a 
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generally asymmetrical cross section. As a further alternative, the electromagnetic radiation 
beam emitter is operative to provide at least one beam having a generally pyramidal shape. 
As yet a further alternative, the electromagnetic radiation beam emitter is operative to 
provide at least one beam having a generally polygonal cross section. 

In accordance with a further preferred embodiment of the present invention the 
electromagnetic radiation beam emitter is operative to provide a modulated beam. 
Preferably, the electromagnetic radiation beam emitter is operative to provide a beam of 
visible light. Alternatively, the electromagnetic radiation beam emitter is operative to 
provide a beam of non- visible electromagnetic radiation. 

In accordance with a still further preferred embodiment of the present invention 
the interface apparatus also includes detector output processing circuitry operative to 
receive at least one output of the at least one detector and to provide an output indication of 
at least one of location, orientation, shape and size of at least one impingement spot defined 
by impingement of the at least one electromagnetic radiation beam on the panel. Preferably, 
the interface apparatus also includes detector output processing circuitry operative to 
receive at least one output of the at least one detector and to provide an output indication of 
at least two of location, orientation, shape and size of at least one impingement spot defined 
by impingement of the at least one electromagnetic radiation beam on the panel. 

In accordance with yet a further preferred embodiment of the present invention the 
interface apparatus also includes detector output processing circuitry operative to receive at 
least one output of the at least one detector and to provide an output indication of at least 
three of location, orientation, shape and size of at least one impingement spot defined by 
impingement of the at least one electromagnetic radiation beam on the panel. Preferably, 
the interface apparatus also includes detector output processing circuitry operative to 
receive at least one output of the at least one detector and to provide an output indication of 
location, orientation, shape and size of at least one impingement spot defined by 
impingement of the at least one electromagnetic radiation beam on the panel. 

In accordance with another preferred embodiment of the present invention the 
interface apparatus also includes detector output processing circuitry operative to receive at 
least one output of the at least one detector and to provide an output indication of at least 
one of the location and angular orientation of the electromagnetic radiation beam emitter. 
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Preferably, the interface apparatus also includes detector output processing circuitry 
operative to receive at least one output of the at least one detector and to provide an output 
indication of the location and angular orientation of the electromagnetic radiation beam 
emitter. 

5 In accordance with yet another preferred embodiment of the present invention the 

interface apparatus also includes detector output processing circuitry operative to receive at 
least one output of the at least one detector and to provide an output indication of at least 
one of the location and angular orientation of the electromagnetic radiation beam emitter, 
the location being defined as a Z-distance between a plane of the panel along a line 

1 0 perpendicular thereto and a plane parallel' to the plane of the panel in which a beam origin 
of the electromagnetic radiation beam emitter is located. Preferably, the interface apparatus 
also includes detector output processing circuitry operative to receive at least one output of 
the at least one detector and to provide an output indication of at least one of the location 
and angular orientation of the electromagnetic radiation beam emitter, the location being 

1 5 defined as a point-to-point distance between a beam origin of the electromagnetic radiation 
beam emitter and a center of an impingement location of the beam on the panel. 

In accordance with an additional preferred embodiment of the present invention 
the interface apparatus also includes detector output processing circuitry operative to 
receive at least one output of the at least one detector and to provide an output indication of 

20 a trajectory of the electromagnetic radiation beam emitter. Preferably, the impingement of 
the beam on the panel provides a generally elliptical impingement spot. Additionally, the 
interface apparatus also includes analysis circuitry operative to determine a ratio of a major 
axis and a minor axis of the elliptical impingement spot, thereby to determine an angle of 
intersection between the beam and the panel. Additionally or alternatively, the interface 

25 apparatus also includes analysis circuitry operative to employ detected variations in 
intensity of the electromagnetic radiation at different locations on an impingement spot 
defined by impingement of the beam on the panel, thereby to assist in determination of an 
angle of intersection between the beam and the panel. 

There is also provided in accordance with another preferred embodiment of the 

30 present invention an interface method including providing a panel defining at least one 
edge, at least one detector arranged along the at least one edge of the panel and an 



WO 2005/094176 



PCT/IL2005/000365 



electromagnetic radiation beam emitter operative to direct at least one beam of 
electromagnetic radiation onto the panel from a variable distance and at a variable angle, 
directing the beam of electromagnetic radiation from the electromagnetic radiation beam 
emitter onto the panel, thereby producing at least one impingement spot, employing the 
panel to transmit electromagnetic radiation from the at least one impingement spot to the at 
least one edge thereof, the panel being operative to generally attenuate the electromagnetic 
radiation passing therethrough to the at least one edge as a function of the distance traveled 
by the electromagnetic radiation through the panel, detecting, by the at least one detector, 
the electromagnetic radiation transmitted by the panel to the at least one edge, and 
employing an output of the at least one detector to determine the variable distance and the 
variable angle. 

In accordance with a preferred embodiment of the present invention providing the 
panel includes providing a display. Preferably, providing the panel includes providing a 
mobile telephone display panel. Alternatively, providing the panel includes providing a 
hand-held computing device display panel. As a further alternative, providing the panel 
includes a providing television display panel. As a still further alternative, providing the 
panel includes providing an input pad panel. 

In accordance with another preferred embodiment of the present invention 
providing the at least one detector includes providing a generally linear array of detectors. 
Preferably, the detecting by the at least one detector includes detecting electromagnetic 
radiation at predetermined frequencies in at least one of visible and non-visible ranges. 
Additionally or alternatively, providing the electromagnetic radiation beam emitter includes 
providing and electromagnetic radiation beam emitter which is operative to provide a 
generally conical beam. 

In accordance with yet another preferred embodiment of the present invention 
providing the electromagnetic radiation beam emitter includes providing and 
electromagnetic radiation beam emitter which is operative to provide a plurality of beams. 
Preferably, providing the electromagnetic radiation beam emitter includes providing and 
electromagnetic radiation beam emitter which is operative to provide at least one generally 
collimated beam. Alternatively, providing the electromagnetic radiation beam emitter 
includes providing and electromagnetic radiation beam emitter which is operative to 
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provide at least one beam having a generally asymmetrical cross section. As a further 
alternative, providing the electromagnetic radiation beam emitter includes providing and 
electromagnetic radiation beam emitter which is operative to provide at least one beam 
having a generally pyramidal shape. As still another alternative, providing the 
electromagnetic radiation beam emitter includes providing and electromagnetic radiation 
beam emitter which is operative to provide at least one beam having a generally polygonal 
cross section. 

In accordance with a further preferred embodiment of the present invention 
providing the electromagnetic radiation beam emitter includes providing and 
electromagnetic radiation beam emitter which is operative to provide a modulated beam. 
Preferably, providing the electromagnetic radiation beam emitter includes providing and 
electromagnetic radiation beam emitter which is operative to provide a beam of visible 
light. Alternatively, providing the electromagnetic radiation beam emitter includes 
providing and electromagnetic radiation beam emitter which is operative to provide a beam 
of non-visible electromagnetic radiation. 

In accordance with a still further preferred embodiment of the present invention 
the interface method also includes providing detector output processing circuitry operative 
to receive at least one output of the at least one detector and to provide an output indication 
of at least one of location, orientation, shape and size of at least one impingement spot 
defined by impingement of the at least one electromagnetic radiation beam on the panel. 
Preferably, the interface method also includes providing detector output processing circuitry 
operative to receive at least one output of the at least one detector and to provide an output 
indication of at least two of location, orientation, shape and size of at least one 
impingement spot defined by impingement of the at least one electromagnetic radiation 
beam on the panel. 

In accordance with yet a further preferred embodiment of the present invention the 
interface method also includes providing detector output processing circuitry operative to 
receive at least one output of the at least one detector and to provide an output indication of 
at least three of location, orientation, shape and size of at least one impingement spot 
defined by impingement of the at least one electromagnetic radiation beam on the panel. 
Preferably, the interface method also includes providing detector output processing circuitry 
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operative to receive at least one output of the at least one detector and to provide an output 
indication of location, orientation, shape and size of at least one impingement spot defined 
by impingement of the at least one electromagnetic radiation beam on the panel. 
Additionally or alternatively, the interface method also includes providing detector output 
5 processing circuitry operative to receive at least one output of the at least one detector and 
to provide an output indication of at least one of the location and angular orientation of the 
electromagnetic radiation beam emitter. 

In accordance with another preferred embodiment of the present invention the 
interface method also includes providing detector output processing circuitry operative to 

1 0 receive at least one output of the at least one detector and to provide an output indication of 
the location and angular orientation of the electromagnetic radiation beam emitter. 
Preferably, the interface method also includes providing detector output processing circuitry 
operative to receive at least one output of the at least one detector and to provide an output 
indication of at least one of the location and angular orientation of the electromagnetic 

15 radiation beam emitter, the location being defined as a Z-distance between a plane of the 
panel along a line perpendicular thereto and a plane parallel to the plane of the panel in 
which a beam origin of the electromagnetic radiation.beam emitter is located. Alternatively, 
the interface method also includes providing detector output processing circuitry operative 
to receive at least one output of the at least one detector and to provide an output indication 

20 of at least one of the location and angular orientation of the electromagnetic radiation beam 
emitter, the location being defined as a point-to-point distance between a beam origin of the 
electromagnetic radiation beam emitter and a center of an impingement location of the 
beam on the panel. 

In accordance with another further preferred embodiment of the present invention 
25 the interface method also includes providing detector output processing circuitry operative 
to receive at least one output of the at least one detector and to provide an output indication 
of a trajectory of the electromagnetic radiation beam emitter. Preferably, the interface 
method also includes providing a generally elliptical impingement spot by impingement of 
the beam on the panel. Additionally the interface method also includes providing analysis 
30 circuitry operative to determine a ratio of a major axis and a minor axis of the elliptical 
impingement spot and employing the analysis circuitry to determine an angle of intersection 
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between the beam and the panel. Additionally or alternatively, the interface method also 
includes providing analysis circuitry operative to employ detected variations in intensity of 
the electromagnetic radiation at different locations on an impingement spot defined by 
impingement of the beam on the panel and employing the analysis circuitry to assist in 
5 determination of an angle of intersection between the beam and the panel. 

BRIEF DESCRIPTION OF THE DRAWINGS 

10 The present invention will be understood and appreciated more fully from the 

following detailed description, taken in conjunction with the drawings in which: 

Figs. 1, 2 and 3 are simplified pictorial illustrations of three alternative 
embodiments of the present invention; 

Figs. 4A 5 4B & 4C are simplified pictorial illustrations of operation of an 
15 embodiment of the present invention implemented in and in association with a mobile 
telephone; 

Figs. 5A, 5B & 5C are simplified pictorial illustrations of operation of an 
embodiment of the present invention implemented in and in association with a personal 
digital assistant; 

20 Figs. 6A, 6B & 6C are simplified pictorial illustrations of operation of an 

embodiment of the present invention implemented in and in association with an input 
tablet; 

Figs. 7 A, 7B & 7C are simplified pictorial illustrations of operation of an 
embodiment of the present invention implemented in and in association with a television; 
25 and 

Fig. 8 is a partially block diagram, partially diagrammatic illustration of 
information extraction circuitry associated with the light pen and light receiving and 
sensing apparatus of Figs. 1 - 3 in accordance with a preferred embodiment of the present 
invention. 

30 
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DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

There are many situations in which wireless remote control of equipment or data is 
particularly desirable. In many of these situations it would be particularly advantageous for 
5 control to be dependent on the spatial positioning or motion of a free moving control device 
relative to a display or panel related to the equipment. 

Three-dimensional remote control maybe particularly advantageous, for example, 
for improving the quality of life of severely disabled people. Such control may be used by 
people with severe mobility impairment, for operating special needs vehicles, household 

10 equipment, computers or the like. Three-dimensional remote control would allow the 
minimum effort with no pressure being required to operate switches or the like. 
Conveniently, the control device could be configured to the needs of the disabled person 
allowing him to wear the electromagnetic radiation-emitting portion of the device in a 
convenient position for mobility. 

15 Three-dimensional remote control also has applications in three-dimensional 

design. It would be particularly beneficial, for example, for a designer to be able to create a 
* virtual' 3D design by tracing a three-dimensional image in mid-air above a control panel in 
the form of a sensor plate. A computer design system integrating a wireless, freely held 
control device in the form of a stylus or the like would allow a more seamless, intuitive, and 

20 artistic effort. 

It is an object of the invention to provide an improved control apparatus for the 
remote control of data or equipment. 

According to one aspect of the invention, there is provided control apparatus for 
the remote control of data or equipment, comprising: a screen; an input device such as a 

25 stylus; sensing means for sensing data related to a position of the input device relative to the 
screen; control means for determining the position of the input device relative to said screen 
from the data; wherein, the input device includes a source of electromagnetic radiation for 
directing onto the screen; the sensing means includes detection means for detecting 
electromagnetic radiation directed onto, and scattered by the screen or by a scattering 

3 0 surface beneath a non-scattering screen; the detection means being configured to produce a 
detector output dependent on the scattered electro-magnetic radiation detected; and the 
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control means being configured to analyze the detector output to determine the position of 
the source relative to the screen. 

Preferably the source is configured to produce a conical beam of infrared radiation 
for directing onto the screen to form an elliptical area of incident radiation. 
5 Alternatively, the source may be configured to produce multiple laser beams for 

directing onto the screen. 

Preferably the control means is configured to analyze the detector output, to 
measure the length of at least one of a major axis or a minor axis of the incident elliptical 
area, and to calculate the distance between the source and the screen from said measured 
10 length. 

Preferably the control means is configured to analyze the detector output to 
measure angle-related parameters of electromagnetic radiation directed by the source onto 
the screen, and to determine the angle of the source relative to screen from the measured 
parameters. 

1 5 Preferably the area-related parameters include the length of at least one of a maj or 

axis or a minor axis of the incident elliptical area. 

Preferably the source is configured to split the laser beam into a plurality of 
components for directing onto the screen to form a plurality of corresponding incident 
points. 

20 Preferably the control means is configured to analyze the detector output to 

identify the surface pattern created by the incident points. 

The control means may be configured to determine the distance of the source from 
the screen from the surface pattern. 

The control means may be configured to determine the angle of the source relative 
25 to the screen from the surface pattern. 

Preferably the detection means comprises a plurality of detectors arranged along at 
least one edge of the screen; each detector being configured to produce an analog output 
dependent on the quantity of scattered electromagnetic radiation detected by it. Typically, 
the detection means' reception frequency corresponds to that of the emitter. 
3 0 Preferably the detection means is configured to produce detector output in the form 

of a digital signal, and the detection means further comprises a converter configured to 
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convert the analog outputs into the digital signal. 

According to another aspect of the invention, there is provided a method of 
determining a position of an input device relative to a screen comprising: directing 
electromagnetic radiation onto the screen from the input device; detecting electromagnetic 
5 radiation directed onto, and scattered by the screen; producing a detector output dependent 
on scattered electromagnetic radiation detected; and analyzing the detector output to 
determine the position of the input device relative to the screen. 

Electronic devices which may embody the present invention may be, for example, 
a hand-held or "palm-top" computer, a personal digital assistant (PDA), a PC screen, or a 

1 0 mobile communication device such as a mobile telephone. Alternatively the device may be 
an electronic control panel for controlling electronic equipment such as surgical apparatus, 
an electric vehicle or the like. Such devices each preferably include a screen for displaying 
data, or control options for controlling electronic equipment. In operation, the data 
displayed or the electronic equipment is controlled by means of an input device. In the 

15 preferred embodiment the input device takes the form of a pointer, which may be of any 
suitable form, for example a pen-shaped "stylus", a glove, a visor, a gun, or any suitable 
remote unit. The pointer allows the user to operate the device by selecting various options 
displayed on the screen. 

The screen preferably is made of a relatively transparent material with small but 

20 significant scattering properties, for example, commercial grade Perspex or the like. Hence, 
in operation, when the pointer emits a beam of electromagnetic radiation onto a region of 
the screen, most of the electromagnetic radiation incident on the screen is transmitted 
through the screen, and some of the electromagnetic radiation is scattered towards the edges 
of the screen. The quantity of transmitted electromagnetic radiation passing through the 

25 screen typically decreases exponentially with the thickness of the screen. The scattered light 
arriving at the edges of the screen spreads over an increasing area with increasing distance 
from the region of incidence to the edges of the screen. The light arriving at an edge at 
distance L from the region of incidence is proportional to (exp -aL)/L. a is the scattering 
coefficient of the screen. Delete corresponding sentence in text. 

3 0 Preferably associated with the screen are a plurality of detectors arranged along the 

edges of the screen in the x and y directions respectively. The detectors are configured to 
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detect the scattered electromagnetic radiation, and typically have a relatively narrow field of 
view, for example -8°. They may be of any form suitable for detecting electromagnetic 
radiation of the type emitted by the pointer, and may be, for example, conventional photo 
detectors. 

5 The pointer is provided with a source of electromagnetic radiation, for example an 

infrared emitter, a laser, an LED or other such light-emitting device. 

According to one embodiment, the pointer emits a conical beam of 
electromagnetic radiation from a circular, spherical, or other shaped tip (not shown). The 
light preferably is modulated in order to avoid interference from ambient light. 
10 In operation, when the pointer emits a beam of electromagnetic radiation onto a 

region of the screen, each detector detects a portion of the electromagnetic radiation 
scattered from the incident beam and produces a corresponding analog output with a 
magnitude dependent on the quantity of scattered electromagnetic radiation detected. The 
analog outputs preferably are received by a converter, which produces a corresponding 
15 serial output. The serial output is inputted to a device which analyses the signal to 
determine the emitter's electromagnetic radiation propagation pattern upon the screen and 
thereby derive the pointer parameters, for example the relative X, Y and Z position and/or 
angle of inclination of the pointer. 

The converter is of any suitable type and may be an analog-to-digital converter. 
20 The serial output may be in the form of a digital serial signal or an analog serial signal. 

The x and y detectors with the highest magnitude analog output generally 
correspond respectively to tiie x and y positions of the center of the incident beam. Hence, 
in operation, the electronic device measures the magnitude of the analog outputs, 
determines the x and y position of the incident beam from the measured values, and 
25 responds appropriately. 

In operation, the accuracy of the position determined may be improved by 
interpolation between the values of the outputs measured for multiple detectors. In each 
direction, the value measured for the detector with the highest magnitude output is 
interpolated with the values measured for adjacent detectors using a polynomial function. 
30 The polynomial function is quadratic having the form: 
V(x)-ax 2 + bx + c for the x direction 
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and 

V(y) = a'y 2 + b'y + C for the y direction 

where the functions V(x) and V(y) are the values of the outputs of the detectors at 
position x and position y respectively. The constants a, a 5 , b, b' and c, c' are determined by 
finding the best fit to the measured values. The interpolated best value is approximated by: 

x = -b/2a for the x position 

and 

y = -b72a' for the y position 

Hence, a more accurate value of x and y position are found. 
It is appreciated that the quadratic model described hereinabove is only one of a 
variety of models. 

The region of the screen on which the electromagnetic radiation is incident will be 
substantially elliptical with an area dependent upon the distance of the pointer from the 
screen. The eccentricity of the ellipse will depend on the angle at which the beam strikes the 
screen and hence on the angle at which the pointer is held. An eccentricity of 0 5 for 
example, is indicative of a circle of incident electromagnetic radiation and the pointer being 
held perpendicular to the surface of the screen. 

The relative magnitudes of the analog outputs are used to determine the lengths of 
the major and minor ellipse axes and hence the eccentricity of the ellipse. The angle of the 
pointer is then calculated using conventional mathematics. Similarly the parameters a and 
a' of the parabolas defined hereinabove are inversely proportional to the ellipse size, and 
are used to calculate the perpendicular distance of the pointer from the screen in the Z 
direction. Where the ellipse is a circle, the length of both ellipse axes will be equal, 
corresponding to the diameter d of a circle. 

The ratio of the constants a to a 9 derived from the interpolation is also a measure 
of the angle of the pointer relative to the screen. Hence, in operation the constants derived 
from interpolation may also be used to improve the accuracy of the angle determined. 

The intensity of the electromagnetic radiation incident on the screen is also 
dependent on the distance of the pointer from the screen. Correspondingly, the quantity of 
scattered electromagnetic radiation detected by the detectors, and hence the magnitudes of 
the analog signals vary with the intensity of the incident electromagnetic radiation. Hence, 
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the absolute magnitudes of the analog signals may also be used, either to independently 
determine the line-of-sight distance of the pointer from the screen, or to refine the result of 
the calculation based on the ellipse measurements. 

The distance of the pointer from the screen is also generally inversely proportional 
to the constants a and a' derived from the interpolation. Hence, in operation the constants 
derived from interpolation may also be used to improve the accuracy of the z position 
determined. 

The location and angle of the pointer may also be used to determine when the user 
makes a selection without physical contact between the pointer and the screen. A simple 
dipping motion, for example, could be used to represent the selection. Alternatively or 
additionally, the area and / or intensity of the light may also be used to represent a 
contactless selection. Such a selection may be indicated, for example, by the area of 
incident electromagnetic radiation falling below a certain minimum threshold and / or the 
intensity rising above a certain maximum threshold. 

According to another embodiment of the invention, the pointer includes a laser 
source, three beam splitters and a window. The laser source may be of any suitable type, for 
example, a laser diode or the like. 

The laser source preferably is located within the pointer and is configured to 
produce a laser beam in an axial direction. The beam splitters are preferably in the form of 
planar, semi-reflecting mirrors arranged within the pointer at intervals along the operational 
path of the laser beam. Each beam splitter is positioned at an angle a/2 relative to the axial 
direction of the laser beam. Hence, in operation each beam splitter splits the laser beam into 
a first component, which passes through the splitter with no change in direction, and a 
second component, which is deflected through an angle a. 

The first, second and third beam splitters are located progressively further from the 
laser source. The second and third beam splitters are respectively rotated through 120°, and 
-120°, relative to the first beam splitter about the axial direction of the laser beam. Hence, 
in operation the laser source produces the laser beam, which passes through each beam 
splitter, in turn. Thus, the beam is split into four components comprising three deflected 
components and a transmitted component. 

In operation, the deflected components incident on the screen, form the vertices of a 
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triangle. The incident transmitted component lies substantially at the centre of the triangle. 
Each beam splitter may be made of any suitable material, for example, microscope cover 
glass. The splitters preferably are each provided with a thin-film optical interference coating 
for controlling the relative transmittance and reflectance of each splitter and hence the 
5 relative intensity of the deflected and transmitted components. Alternatively or additionally 
the pointer may be provided with an absorbing filter located in the operational path of the 
transmitted component for attenuating the intensity of the transmitted component relative to 
the deflected components. 

Alternatively, a diffractive screen may replace the mirror assembly. 
10 Thus, in operation, the intensity of the transmitted component is different to the 

intensity of the deflected components. Hence, when the components of the laser beam are 
incident on the screen, the transmitted component can be distinguished from the deflected 
components. 

By way of example, when the beam splitters are made of microscope cover glass 
15 and a is set to approximately 13°, the associated ratio of transmittance to reflectance is 
approximately 1:1. Hence, the intensity of the transmitted component emerging from the 
third beam splitter is approximately: 25% of the intensity of the first deflected component 
deflected by the first beam splitter; 50% of the intensity of the second deflected component 
deflected by the second beam splitter and equal to the intensity of the third deflected 
20 component deflected by the third beam splitter. The absorbing filter reduces the intensity of 
the transmitted component further so that it is less than that of the other components. 

A window may be located at the end of the pointer in the operational path of the 
transmitted component of the laser beam. The window is of suitable size and configuration 
to allow passage of all three deflected components and the transmitted component during. 
25 operation. 

The use of a laser source is particularly advantageous for operation of the device 
from a distance, since laser beams have extremely narrow spreads and maintain a relatively 
high intensity over long distances. It will be appreciated that the precise location and 
arrangement of the laser source and beam splitters may vary dependant on requirements. 
30 For example, where remote operation is required from a distance, the angle a may be 
particularly acute. 

15 



WO 2005/094176 



PCT/IL2005/000365 



Operation to determine the three-dimensional position and orientation of the 
pointer employing multiple laser beams is similar to that described for the pointer having a 
conical beam. 

The distance between the transmitted and deflected components incident on the 
5 screen is determined from their relative positions and hence the distance in the Z direction 
can also be determined by conventional trigonometry. 

The relative distances between the transmitted and deflected components incident 
on the screen is indicative of the equality of the triangle formed by the incident deflected 
components. For example, when the pointer is held with the transmitted component 

1 0 perpendicular to the screen, the triangle formed by the deflected components is substantially 
equilateral. When the transmitted component is not normal to the screen, the triangle 
formed is not equilateral and the distances between the components reflect this. Hence, the 
angle at which the pointer is being held relative to the screen is determined using 
conventional trigonometry. 

1 5 Repetitive calculation of the pointer position several times a second as the pointer 

is moved, allows a pointer trajectory to be recorded. The pointer trajectory may then be 
used to assist in anticipating the intentions of the user or for advanced control functions. 

The location and angle of the pointer may also be used to determine when the user 
makes a selection without physical contact between the pointer and the screen. A simple 

20 dipping motion, for example, could be used to represent the selection. 

It will be appreciated that although the invention has been described hereinabove 
with reference to an embodiment in which the three-dimensional position of a pointer may 
be determined, the invention is equally applicable for determining the position of the 
pointer in two dimensions only, or even a single dimension. 

25 It will further be appreciated that although the invention has been described with 

reference to particular applications, there are numerous other ways in which the invention 
may be applied. A number of applications will now be described for the purposes of 
illustration. 

The invention has particular application in the medical profession, for example, for 
30 computer-aided surgical procedures such as laparoscopy, microsurgery, and endoscopy. In 
these cases, the light source could be built into tip of a pointer in the form of a scalpel or 
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other tool for a more natural feel for a doctor or surgeon. 

In telemedicine the invention may be used for the control of cameras and medical 
equipment in remote medical examinations or procedures. 

The invention may also be incorporated into equipment for training purposes. For 
5 example, it would be particularly beneficial in medical student Ixaining, for allowing 
simulated medical procedures to be carried out on computerized dummies such as CPR 
training mannequins or using other computer controlled simulators. A pointer in the form of 
a light-tipped surgical tool could be used to enhance the realism of the simulation making it 
more effective. Potentially such use could also reduce the requirements for cadavers in 

1 0 medical training. 

The invention would also be of advantage in applications where there is a 
requirement for the human operation of equipment based in hazardous environments such 
as the vacuum of space or military settings. Examples of such uses include the intuitive 
control of unmanned equipment and the external repair and adjustment of space-based 

15 vehicles and equipment. A single hardware system could therefore be programmed with 
separate programming preferences for each piece of equipment, minimizing weight and 
equipment limitations on board. 

The invention also has many applications in small-scale research, engineering and 
manufacturing including biotechnology, biology, semiconductors and nanotechnology. In 

20 situations where the operator is working with items too small to be seen by the naked eye, a 
system can be devised to integrate the pointer with a computerized microscope, for 
example, an electron-microscope or the like. The microscopic real-time image can then be 
used in conjunction with the pointer to control the microscope and to guide manipulate a 
6 workpiece' . In this application it would be particularly beneficial if the light source were a 

25 laser, since this can be focused extremely precisely. 

Beneficial functionality may include, for example, simultaneous zoom control in 
conjunction with active functions in a single wireless pointer in the form of a stylus or other 
tool, the pointer being used directly "on" a microscope image for designing microchips, or 
investigating viruses or genes. 

30 The invention may also be incorporated into equipment for fertility treatments, 

such as ICSI (intracytoplasmic sperm injection). In ICSI, a microscopically tipped pipette is 
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used to select a single sperm and to inject it into an egg. Having the pipette controlled by an 
operator holding a pointer in the form of a computer-driven, light-tipped "pipette" and 
working directly on the microscope-generated image would have obvious advantages. The 
same principle applies to ex-vivo treatment (IVF), colonoscopy, and other invasive biopsy 
investigations, where trauma to a patient needs to be minimized. 

In all these situations, the pointer is a hand-held tool which does not come into 
direct contact with the work in progress. This allows appropriate preferences and tolerances 
to be set to reduce the probability and magnitude of errors caused by tinder-compensation, 
over-compensation or undesirable movements. 

A further application of the invention is for the enhanced control of televisions and 
other entertainment equipment. An electromagnetic radiation-sensitive screen in 
combination with an appropriately configured pointer could be used in a way similar to 
current remote-control systems, but with additional enhancements. For example, 
adjustability of a "frame within a frame" could be provided with "click and drag" being 
used to adjust the size of the second frame. Similarly, provision could be made for direct 
on-screen programming of a VCR, other recording device or integrated media. 

A particular advantage in these applications is the provision of button-free menu 
operation, allowing for a more user-friendly and truly interactive procedure. 

Reference is now made to Figs. 1, 2 and 3, which are simplified pictorial 
illustrations of three alternative embodiments of the present invention. Figs. 1, 2 and 3, 
each show a panel 100 5 preferably a generally planar element which is preferably but not 
necessarily part of a display screen. For example, the panel 1 00 may be a plastic protective 
cover over a solid state display, such as a liquid crystal or VGA display in a cellular 
telephone or PDA, a portion of a touch-screen or a phosphor-backed glass panel of a CRT 
display. 

Panel 1 00 may be made of any suitable material, and is typically formed of a glass 
or plastic substrate, such as Plexiglas and need not necessarily be rigid. Alternatively, a 
transparent non-scattering element in contact with aLambertian surface may be employed. 
Panel 100 is preferably characterized in that it transmits electromagnetic radiation 
impinging thereon to edges thereof. The panel generally attenuates the electromagnetic 
radiation passing therethrough to the edges thereof as a function of the distance traveled by 
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the electromagnetic radiation through the panel. 

Provided along at least one and preferably all edges of panel 100 are a plurality of 
electromagnetic energy detectors 102, such as PIN photodiodes, which are preferably 
arranged in spaced linear arrays, here generally designated by reference numerals 1 04, 1 06, 
108 and 110 disposed along respective edges 114, 116, 118 and 120 ofpanel 100. Portions 
of edges 11 4, 116, 118 and 120 may be covered or coated other than at detectors 102, to 
help define and narrow the fields of view of the individual detectors 102. 

Detectors 1 02 are preferably selected such that each has a field of view having an 
angular region of coverage of approximately "8 degrees, and such that if necessary, a part of 
the region of coverage of each detector can be masked, in order to achieve reduction in 
overlap between regions of coverage. Detectors 102 are preferably arranged and spaced 
such that generally the entire area of the panel 1 00 is covered by the field of view of at least 
one detector 1 02. The fields of view of a few of detectors 1 02 are illustrated in dashed lines 
and designated by reference numeral 122. 

By virtue of the distance dependent electromagnetic radiation attenuation provided 
by panel 100, the relative amplitudes of the outputs of the detectors along multiple edges 
can be used to provide an output indication of the impingement location of a beam of light 
on the panel 100. The amplitudes of the outputs of the detectors can also be employed to 
indicate the area of a beam of electromagnetic radiation impinging on the panel, as well as 
its intensity. 

The outputs of each of the detectors 102 are preferably multiplexed by detector 
output multiplexing circuits. It is appreciated that in certain applications, detectors 1 02 need 
not be provided along each edge of the panel, but it may be sufficient for detectors 102 to 
be provided along two generally orthogonal edges thereof Preferably, however, detectors 
102 are provided along all edges of the panel in order to maximize the signal-to-noise 
performance of the device. 

A light pen or other suitable electromagnetic radiation emitter is employed to 
produce a beam of electromagnetic radiation which impinges on the panel 1 00. As will now 
be described, the location of the light pen and the angular orientation of the beam of 
electromagnetic radiation produced thereby relative to the panel can be determined from the 
outputs of the detectors. 
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The location of the light pen may be expressed as a Z-distance between a plane of 
the panel 100 along a line perpendicular thereto and a plane parallel to the plane of the 
panel in which the tip of a light pen is located. The location of the light pen may also be 
expressed as a point-to-point distance between the tip of the light pen and the center of the 
5 impingement location of the electromagnetic radiation beam on the panel 1 00. 

Turning to Fig. 1, a light pen 150 is shown emitting a generally conical beam 152 
which impinges on panel 1 00 and defines an impingement spot 1 54 thereon. As seen in Fig. 
1 , the electromagnetic radiation beam 1 52 has a beam axis 1 56 which is angled with respect 
to a generally planar element 158 by a non-90 degree angle. The impingement spot 154 

1 0 forms generally an ellipse whose major axis and minor axis intersect at the intersection of 
beam axis 156 and generally planar element 158. Were the beam axis 156 to be arranged at 
a 90-degree angle to the generally planar element 158, the impingement spot 154 would 
have the shape of a circle, which is a degenerate case of an ellipse, and the center of the 
circle would lie at the intersection of beam axis 156 and generally planar element 158. 

15 It is a particular feature of the present invention that the outputs of detectors 102 

on at least two orthogonal edges 114 and 116 and preferably the outputs of detector arrays 
104, 106, 108 and 110 along respective orthogonal edges 114, 116, 118 and 120 provide 
information as to the location, orientation, shape and size of the impingement spot 154. 

As described hereinbelow with reference to Fig. 8, the processing circuitry of the 

20 present invention preferably provides combined outputs of detectors lying along each edge 
of the panel. These outputs can be employed to provide information regarding location, 
orientation, shape and size of impingement spot 154 on panel 100 produced by 
impingement thereon of beam 152 of electromagnetic radiation produced by an 
electromagnetic radiation emitter 150. 

25 It is appreciated that the outputs of detectors 102 provide information to 

distinguish between impingement spots having different locations, orientations, shapes and 
sizes. For example, when comparing impingement spots 154 and 182 in Fig. 1, it is 
appreciated that a change in the size of the spot produces a corresponding change in the 
combined outputs of detectors 1 02, such as in the breadths and peak intensities thereof. As 

30 another example, when comparing impingement spots 154 and 184 in Fig. 1, it is 
appreciated that a change only in the location of the spot produces a corresponding change 
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in the combined outputs of detectors 102, such as in the location of the center of the peak 
thereof and in the relative intensities sensed by detectors located at opposite edges of the 
panel 100. 

As a further example, when comparing impingement spots 1 54 and 1 86 in Fig. 1 , it 
is appreciated that a change in the shape of the spot produces a corresponding change in the 
combined outputs of detectors 102, such as in the relative breadths and intensities thereof. 
As an additional example, when comparing impingement spots 154 and 188 in Fig. 1, it is 
appreciated that a change in the orientation of the spot produces a corresponding change in 
the combined outputs of detectors 1 02, such as in the relative breadths, peak intensities and 
symmetries as sensed by detectors located at opposite edges of the panel 100. 

It is appreciated that the different intensities of radiation at different sides of the 
ellipse are indicative of the angular orientation of the beam 152, and may be used to 
calculate the angle at which beam is directed at panel 100. 

It is appreciated that variations in location, orientation, shape and size of the 
impingement spot correspond to variations in the three-dimensional orientation and three- 
dimensional location of the light pen 150. This is due, inter alia, to the facts that the ratio of 
the lengths of the major and minor axes of an ellipse are unique to an angle of intersection 
of a plane and a cone, and that for each such angle, the size of the ellipse is related to the 
distance between the vertex of the cone and the plane. 

Turning to Fig. 2, a light pen 240 is shown emitting a plurality of generally 
collimated beams 250, 25 1 , 252 and 253, such as non-mutually parallel laser beams, which 
impinge on panel 1 00 and define an impingement pattern 254 thereon. As seen in Fig. 2, the 
light beams 250, 251, 252 and 253 are directed non-perpendicularly with respect to a 
generally planar element 258. The impingement pattern 254 defines the vertices and the 
center of a triangle, the center of the triangle being defined by impingement of beam 250 
onto generally planar element 258. Were the beam 250 to be arranged at a 90-degree angle 
to the generally planar element 25 8 , the impingement pattern 254 would define the shape of 
an equilateral triangle. 

It is a particular feature of the present invention that the outputs of detectors 102 
on at least two orthogonal edges 114 and 116 and preferably the outputs of detector arrays 
104, 106, 108 and 110 along respective orthogonal edges 1 14, 1 16, 1 18 and 120, provide 



21 



WO 2005/094176 



PCT/IL2005/000365 



information as to the location, orientation, shape and size of the triangle defined by the 
impingement pattern 254. 

It is appreciated that the outputs of detectors 102 provide information to 
distinguish between the impingement patterns having different locations, orientations, 
5 shapes and sizes. It is appreciated that a change in any one or more of location, orientation, 
shape and size of the pattern produces a corresponding change in the combined outputs of 
detectors 102, such as in the distributions and configurations thereof. 

It is appreciated that variations in location, orientation, shape and size of the 
impingement pattern, correspond to variations in the three-dimensional orientation and 

10 three-dimensional location of the light pen 240. This is due, inter alia, to the fact that the 
geometrical relationship of the vertices of the triangle correlate with an angle of intersection 
of the collimated beams and the panel 200. The size of the triangle is related to the distance 
between the emission point of the electromagnetic radiation and the plane of the panel. 

Turning to Fig. 3, a light pen 350 is shown emitting an asymmetrical, generally 

1 5 triangular pyramidal beam 3 52 which impinges on panel 1 00 and defines an impingement 
spot 354 thereon. As seen in Fig. 3, the light beam 352 has a beam axis 356 which is angled 
with respect to a generally planar element 3 5 8 by a non-90 degree angle. The impingement 
spot 354 is a triangle whose center lies at the intersection of beam axis 356 and generally 
planar element 358. Were the beam axis 356 to be arranged at a 90-degree angle to the 

20 generally planar element 358, the impingement spot 354 would have a predetermined 
desired triangular shape and the center of the triangle would lie at the intersection of beam 
axis 356 and generally planar element 358. 

It is a particular feature of the present invention that the outputs of detectors 102 
on at least two orthogonal edges 114 and 116 and preferably the outputs of detector arrays 

25 104, 106, 108 and 110 along respective orthogonal edges 1 14, 1 16, 1 18 and 120, provide 
information as to the location, orientation, shape and size of the impingement spot 354. 

It is appreciated that the outputs of detectors 102 provide information to 
distinguish between the impingement spots having different locations, orientations, shapes 
and sizes. It is appreciated that a beam having asymmetrical properties maybe employed in 

30 order to further enhance the detectability of changes in angular orientation of an 
electromagnetic radiation emitter. 
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Reference is now made to 4A, 4B and 4C which illustrate operation of an 
embodiment of the present invention implemented in and in association with a mobile 
telephone. In the illustrated embodiment, a light pen 500 providing a generally conical 
beam is employed, it being appreciated that any other suitable beam configuration may be 
5 employed. Figs. 4A - 4C show that the user interface of the present invention is responsive 
to the distance of the light pen 500 from a display panel 502 of a mobile telephone 504 by 
sensing the size of an impingement spot 506 thereon, which size is directly related to the 
distance. 

Fig. 4A shows the light pen 500 being held relatively far from the display panel 
10 502 and providing a relatively large impingement spot 506 which is located on an 
application icon 508, preferably resulting in highlighting that icon. Fig. 4B shows the light 
pen 500 having been moved closer to the display panel 502 and thus providing a relatively 
smaller impingement spot 506 located on the icon 508, preferably providing a zoom-in 
function, resulting in enlarging of the highlighted icon 508. Fig. 4C shows the light pen 500 
1 5 having been moved even closer to the display panel 502 and thus providing an even smaller 
impingement spot 506 located on icon 508, resulting in selection of the application 
represented by the highlighted icon 508 and presenting a menu 510 for operation of the 
application. 

Reference is now made to 5A, 5B and 5C which illustrate operation of an 
20 embodiment of the present invention implemented in and in association with a hand-held 
computing device, such as a personal digital assistant. In the illustrated embodiment, a light 
pen or other electromagnetic radiation beam emitter 520 providing a generally conical beam 
is employed, it being appreciated that any other suitable beam configuration may be 
employed. Figs. 5 A - 5C show that the user interface of the present invention is responsive 
25 to the location of impingement of an electromagnetic light beam produced by a light pen 
520 on a display panel 522 of a personal digital assistant 524 as well as the distance of the 
light pen 520 from the display panel 522, which is determined by sensing the size of an 
impingement spot 526 thereon, which size is directly related to the distance. 

Fig. 5A shows the light pen 520 being held relatively far from the display panel 
30 522 and providing a relatively large impingement spot 526 which is located on an picture 
thumbnail 528, preferably resulting in operation of a moderate zoom-in function on that 
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thumbnail. Fig. 5B shows the light pen 520 having been moved to a different thumbnail 530 
on the display panel 522 preferably providing a moderate zoom-in function on thumbnail 
530. Fig. 5C shows the light pen 520 having been moved closer to the display panel 522 
and thus providing a smaller impingement spot 526 located on thumbnail 530. providing a 
5 maximum zoom-in function on thumbnail 530. 

Reference is now made to 6A, 6B and 6C which illustrate operation of an 
embodiment of the present invention implemented in and in association with a computer 
input device such as an input tablet. In the illustrated embodiment, a light pen or other 
electromagnetic radiation beam emitter providing a generally conical beam is employed, it 

1 0 being appreciated that any other suitable beam configuration may be employed. Figs. 6 A - 
6C show that the user interface of the present invention is responsive to the angular 
orientation an electromagnetic light beam 538 produced by a light pen 540 on a pane! 542 
of an input tablet 544, associated with a computer 546. The angular orientation is 
determined by sensing the eccentricity of an impingement spot 546 on the panel 542, which 

15 eccentricity is directly related to the angular orientation of the beam 538. 

Fig. 6A shows the light pen 540 being held at a first three-dimensional angular 
orientation relative to the panel 542, preferably resulting in presentation of an object 548, 
such as a cylinder, in a first three-dimensional orientation. Fig. 6B shows the light pen 540 
having been moved to a second three-dimensional angular orientation relative to panel 542, 

20 preferably resulting in presentation of obj ect 548 in a second three-dimensional orientation, 
which is preferably rotated relative to the first three-dimensional orientation thereof by an 
amount identical or otherwise correlated to the angular difference between the first and 
second three-dimensional angular orientations of the light pen 540. 

Fig. 6C shows the light pen 540 having been moved to a third three-dimensional 

25 angular orientation relative to panel 542, preferably resulting in presentation of obj ect 548 
in a third three-dimensional orientation, which is preferably rotated relative to the second 
three-dimensional orientation thereof by an amount identical or otherwise correlated to the 
angular difference between the second and third three-dimensional angular orientations of 
the light pen 540. 

30 Reference is now made to 7A, 7B and 7C which illustrate operation of an 

embodiment of the present invention implemented in and in association with an interactive 
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display device, such as a suitably equipped television. In the illustrated embodiment, an 
infra-red emitting remote control device or other suitable electromagnetic radiation beam 
emitter providing a generally conical beam is employed, it being appreciated that any other 
suitable beam configuration may be employed. Figs. 7A - 7C show that the interface of the 
present invention is responsive to the angular orientation an electromagnetic radiation beam 
558 produced by a infra-red emitting remote control device 560 on a panel 562 defined by 
a television screen 564. The angular orientation is determined by sensing the eccentricity of 
an impingement spot 566 on the panel 562, which eccentricity is directly related to the 
angular orientation of the beam 558. 

Fig. 7A shows the infra-red emitting remote control device 560 being held at a 
first three-dimensional angular orientation relative to the panel 562, preferably resulting in 
aiming of an object 568, such as a gun, in a first three-dimensional orientation. Fig. 7B 
shows the infra-red emitting remote control device 560 having been moved to a second 
three-dimensional angular orientation relative to panel 562, preferably resulting in aiming 
of the gun 568 in a second three-dimensional orientation, which is preferably rotated 
relative to the first three-dimensional orientation thereof by an amount identical or 
otherwise correlated to the angular difference between the first and second three- 
dimensional angular orientations of the infra-red emitting remote control device 560. 

Fig. 7C shows the infra-red emitting remote control device 560 having been 
moved to a third three-dimensional angular orientation relative to panel 562, preferably 
resulting in presentation of object 568 in a third three-dimensional orientation, which is 
preferably rotated relative to the second three-dimensional orientation thereof by an amount 
identical or otherwise correlated to the angular difference between the second and third 
three-dimensional angular orientations of the infra-red emitting remote control device 560. 

It is appreciated that the foregoing illustrated functionalities are merely examples 
of possible applications enabled by the present invention. These and other applications may 
be used singly or in any combination in any given device or methodology as appropriate. 

Reference is now made to Fig. 8, which is a partially block diagram, partially 
diagrammatic illustration of information extraction circuitry associated with the 
electromagnetic radiation receiving and sensing apparatus of Figs. 1 - 3 in accordance with 
a preferred embodiment of the present invention. As seen in Fig. 8, the outputs from 
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detectors 602 are multiplexed in one or more multiplexers, here shown as multiplexer 604 
which is operative to sample the outputs of the sensors at a relatively high rate, typically 
about 10 KHz. The output of the multiplexer 604 is preferably supplied, preferably via a 
band pass filter 606, to an integrator 608. The output of the integrator is supplied to a 
microprocessor 610 which is programmed to provide combined outputs representing the 
outputs of detectors along each edge 612 of panel 614. 

The outputs of microprocessor 610 are preferably provided to a microprocessor 
616, which provides outputs representing location, orientation, shape and size of an 
impingement spot 618 on panel 614 produced by impingement thereon of a beam 620 of 
electromagnetic radiation produced by an electromagnetic radiation emitter 622. The 
outputs of microprocessor 6 1 6 are preferably supplied to an additional microprocessor 624, 
which provides outputs representing the distance and angular orientation of the 
electromagnetic radiation emitter 622 in real time. Alternatively, one or more of the 
aforementioned microprocessor functions may be combined or split as appropriate. 

It will be appreciated by persons skilled in the art that the present invention is not 
limited by what has been particularly shown and described hereinabove. Rather, the scope 
of the present invention includes both combinations and subcombinations of the various 
features described hereinabove, as well as modifications and variations thereof which 
would occur to a person of ordinary skill in the art upon reading the foregoing description 
and which are not in the prior art. 
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